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ABSTEACT 

A Fourier teclmique for cloud motion estimation from pairs of pictures is 
described. This technique makes use of the phase of the cross spectral density; 
it allows motion estimates to be made for individual spatial frequencies, wliich 
are related to cloud pattern dimensions. Eesults obtained using this technique 
are presented and are compared with tlie results of a Fourler-domain cross- 
correlation scheme, using both artificial and real cloud data. It is concluded 
that this technique is relatively sensitive to the presence of mixtures of motions, 
changes in cloud shape, and edge effects. 
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A FOURIER APPROACH TO CLOUD MOTION ESTIMATION 
I. INTRODUCTION 

Sequences of high resolution photographs from satellites in geosynchronous 
orbit have been utilized by many investigators to measure cloud motion and de- 
rive winds. The techniques for extracting such information range from a 
manual analysis of closed "movie loops" (Fujita et al. , 1969; Hubert and 
Whitney, 1971) to completely automated methods in which tliC images and/or 
data are computer processed but selection of cloud targets is controlled manu- 
ally from an interactive display console (Serebreny et ah » 1969; Smith and 
Phillips, 1972). 

The main advantage of majuial and semi-automatic methods is in target 
selection, making it possible to incorporate a wide variety of criteria, not 
necessarily defined beforehand, for choosing cloud elements representati'^e of 
atmospheric motion. These criteria are sometimes related to the size of the 
cloud element, the idea being that small elements move with the wind, while 
massive elements may disguise the actual motion of tiae atmosphere due to 
cloud foinnation and deformation processes. 

Early studies by Leese and Epstein (1963) have shown that one can identify 
the predominant dimensions of cloud patterns by a spectral analysis of satellite 
photographs. Subsequently, Leese et al. (1971) used the Fourier transform as 
an efficient computational tool for obtaining the cross-correlation between suc- 
cessive cloud cover pictures, from w'Mch cloud displacements are derived. 
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The fact that displacement information appears separately for each spectral 
component in the Fourier transform of the cross-covai'iance function was 
pointed out by Weinstein (1972), The combined advantages of the Fourier tx-ans- 
form — as an efficient computational tool for measuring displacement and as a 
means for extracting cloud pattex’n dimensional information — have motivated 
the present study of the Fourier ti’ansfoi'ni properties of real and simulated 
sequences of satellite cloud cover pictures aixd their cr~'!s-coi’relations. Spe- 
cifically, we describe an attempt to dex'ive winds from cloud motion, while dis- 
criminating with I’espect to cloud size. 

The xnathematical aspects of the Fourier transfoxm method are outlined in 
Section 2 ; the results of a simulation studj'^ are presented in Section 3; a pre- 
liminary experiment with real pictures is described in Section 4 ; and finally, 
the outlook is discussed and conclusions drawn in Section 5. 

n. PROPERTIES OF THE FOURIER TRANSFORM 

A function defined in the space domahi, say f(x,y), where x and y are co- 
ordinates in a two-dimensional Cai'tesian space, can be ti’ansformed hxto a 
function in the frequency domain by using the Fourier tx'ansfoxm, F, as follows: 

F[f(x,y)l hh r(x,y) dxdy = F(u,v) (^) 


where i u aiid v are frequency domain variables. Conversely, a fmxction 
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in the frequency domain can be transformed into the space domain by using the 
inverse Fourier transform; 


F-' [F(i.,v)] 


II 


F(u,v) dudv = f(x,y) 


( 2 ) 


For application to discrete data, tlie Fourier transform process is re- 
defined as 
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(4) 


where M and N are the sample sizes in the x and y directions, respectively. 

One mdication of the relative displacement between successive pictures is 
by the location of the pealc of the cross -covariance of the two images {Leese et 
al. , 1971). Letting f(x,y) and g(x, y) represent the brightness distribution of 
the two images, the ci’oss-covariance is defined as 

Cfg(x,y) = 

The Foux’ier transfoirm provides a fast way to calculate the cross-covariance 
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r(£.i7)g(f-x,T7-y) d^dT? 


( 5 ) 
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function. From equations (1) and (2), it follows that 

Cf^(x.y) = F-> [F=i'(u,v) • G(u,v)] (6) 

where F*(u,v) is the complex conjugate of F(u,v) while G(u,v) is tlie Foiu’ier 
transform of g'(x,y). The Fourier transform of Cf^, namely F*G, is called tlie 
cross spectral density of f and g. The positions of the maxima of the cross- 
covariance function determine tlie relative displacement between the two pic- 
tures. The ratio of the computer thnes required for the Fourier transfoi'm 
metliod and the conventional method of calculating the cross-covariaiice is ap- 
proximately {N/2)N (Brigham and Morrow, 196 7), where N is the linear size 
of the array. 

If tlie discrete version of the Fourier transform is used to compute the 
cross -CO variance, as in (6), the resulting function is cyclic, which is equivalent 
to assuming that the fimctions in tlie space domain are periodic. In other words, 
the Fourier transform metliod assumes that when movement of objects takes 
place, what moves out of the picture at one boimdai'y must entei’ at the opposite 
boundary. Further examination and discussion of this assumption is presented 
in Section m. 

The Fourier transform possesses still another property which maizes it 
possible to talce a different approach to tlie estimation of motion. Assiune that 
uniform translation is the only difference between tlie two functions, i. e. , 
g(x,y) = f(x-a, y-b). Then from tlie definition of the Fourier transform in 


5 


equation (1) it follows (Briggs, 1968) that 


G(u,v) = F(li,v) c-2"Kua+vb) 

The Fourier toansform at a specific frequency (u, v) can be represented by 
its amplitude and phase angle in the following manner; 


F{u,v) = lF(i.i,v) I 


( 8 ) 


where |F! is the amplitude and i^is the pliase angle. Applying the same repre- 
sentation to G(u, v), we have 


Thus, 


G(ii,v) = [F(u,v) I 


( 9 ) 


F*(u,v)G(u,v) = lF(u,v)l“ (10) 

The term F* (u, v)G(u, v) is complex with amplitude IF I" and phase angle 
-27 t (ua+vb), for all u's and v's. The differences beUveen the phase angles of 
two neighboring frequencies have the following properties: 

tp(u,v) — (p(u+l ,v) = 27ra (11) 


and 


>^(u,v) -^(u,v+I) = 27rb ( 12 ) 

We thus find that the difference in phase angle between neighboring terms in the 
F*G matrix yields the x and y components of displacement. 
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In the case where tlie two images differ only in that one is a uniform trans- 
lation of the other, the phase differences are the same between all neighboxing 
frequencies. However, when the displacement between the two images is not 
unifonn — and, specifically, when disr’acement is different for feature;^ with 
different characteristic dimensions — then the phase angle difference will be a 
function of frequency. In die following sections, v.’s will examine to what extent 
the phase angle differences continue to yield estimates of the displacement and 
how these estimates are related to the size of featui’es in the image. 

ill. SIMULATION STUDIES 

A number of simulation studies were conducted to investigate the compara- 
tive effectiveness of cross-correlation and phase difference techniques for es- 
timatiug motion (Lo and Parikh, 1973; Lo et al. , 1974). In particular, cases 
were studied that involved 

a. Distortions such as rotation, scale change, brightness change, and 
noise, in addition to simple traxxslation 

b. Muctures of motion 

c. Edge effects: objects moving off one boundary without coming back 
in at die opposite boundary, so that the displacements are non-cyclic. 

It \ as found that if these conditions were not too strongly present, both tech- 
niques yielded reasonable estimates of the motion (or, in mixture cases, esti- 
mates of the predominant motion). However, as tlie distortion, mixture, or 
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edge factors become stronger, performance began to deteriorate. This situation 
is illustrated in the simulations shown in Figures 1 and 2. 

In Figure 1, a large mass of relatively dim cloud is moving at a compara- 
tively slow speed (displacement components X = 2 and Y = 1) relative to 
smaller clouds of higher brightness (X = 4, Y = 6). Figure la shows the two 
pictures, Figure lb shows their power spectra, and Figure Ic shows their 
cross-covariance, wliich has a peak corresponding to the displacement (4, 6) 
of the higher-contrast clouds. Figure Id shows the phase angles of the Fourier 
transform product, and Figure le shows the resulting X and Y displacement 
estimates from the phase difference method (obtained by taking horizontal and 
vertical differences in Figure Id). Figure If is a scatter plot of these esti- 
mates, which cluster around the displacement (4, 6). 

Figure 2 is analogous to Figure 1 for a pair of pictures (shown in Figure 2a) 
containing two sets of clouds mth the same brightness but different velocities, 
represented by tlie displacements (6,0) and (2,2), Here the cross-covaiiance 
(Figure 2c) corresponds to the displacement (2,2) of the larger cloud mass; but 
the phase difference estimates (Figure 2e) show little tendency to cluster (Figure 2f). 

It worJd seem from these examples tliat tlie phase difference estimation 
approach is more sensitive to the presence of mixtures of motions than the 
cross-covariance approach. Similar results can be obtained by stud^dng simula- 
tions involving edge effects and other distortions. The details can be found in 
die two references cited at the beginning of the section. 



IV. EJ'PEEIMENTS WITH HEAL DATA 


Several experiments witli pairs of real cloud windows were also carried 
out, as described by Lo et ai. (1974). The windows used were selecved from 
ATS-1 geosynchronous satellite images, taken at intervals of 47 minutes. Each 
wmdow is 64 by 64 pixels, and has been scaled to the gray level range 0 to 63. 

The two windows shown in Figure 3a contain relatively small clouds. In 
order to better evaluate the synoptic environment in which tliese cloud patterns 
are embedded, two larger windows (256 by 256 pixels) containing the smaller 
windows at their centers are shown in Figure 3b. It is evident that the clouds 
in this scene tend to dissipate and re-form relatively fast, so that considerable 
distortion has occurred behveen the frames. Nevertheless, many of the cloud 
patterns in the first window are easily recognizable in the second. Based on 
such patterns, a hand estimate of the relative displacement was made using 
computer printouts of the pictures so that the displacement could be accu- 
rately measured. The estimated displacement, averaged over several cloud 
elements, was X = -5, Y = -3, 

The phase difference method applied to the pair of windows in Figure 3a 
gave rise to a displacement estimate of (-12, -4). Thi*: estimate was ob- 
tained as follows; X and Y displacement estimates were obtained for each 
spatial frequency component. The means and standard deviations of these es- 
timates were computed, and estimates deviating from the mean by more than 
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one standai’d deviation were discarded. The mean was then recomputed for the 
surviving estimates, and was used as the final estimate. 

The cross -covariance method, on the other hand, yielded a displacement es- 
timate of (-6, -3), as determined by the position of the cross-covai'iance peak. 

In this example, therefore, the cross-covariance method yields better results 
(assuming the hand estimate to be most reliable). 

An attempt was made to improve the results obtained from the phase differ- 
ence method by appl 5 dng it to selected bands of spatial frequencies, rather tlian 
to all frequencies. The following results were obtained; 


Band 

Frequency Range 

X 

y 

a 

0 < + v^ < 5 

-8 

2 

b 

5 < + v“ <10 

-13 

16 

c 

10 < V*u^ + v^ <64 

-12 

-4 


Note tliat the last result is the same as that obtained when ail frequencies were 
used, and that those residts still do not agree very well with the hand estimate. 
The cross-covariance estimates were also re-evaluated using only selected 
spatial frequencies (i. e. , tlie cross spectral density F*G was computed, the 
frequencies outside the selected band were suppressed, and the inverse Fourier 
transform of the result was computed to yield a "filtered" cross-covariance). 
The results were as follows : 
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Frequency Band 

X Y 

a 

-6 -5. 5 

b 

-5 -3 

c 

No clearcut peak 


Results (a-b) are quite close to tlie result (-6, -3) obtained when no filtering was 
used, and also agree well with the hand estimate (-5, -3). 

Vo DISCUSSION AND CONCLUSIONS 

The results of the experiments reported here indicate that both the cross - 
covairiance and phase difference methods can be good estimators of motion when 
the objects being tracked do not change tlieir shape, size and orientation to more 
than a limited degree. These teclmiques are less effective when a mixture of 
motions exists, unless one of the motions is strongly dominant. These prop- 
erties indicate that the Fourier transform phase difference estimation methods 
should be reliable in problems such as landmark matclnng, where the features 
do not change appreciably as viewed in satellite photographs. 

In the atmosphere, where die clouds grow, dissipate, rotate, and move 
relative to each others the estimation of these motions using Fourier transform 
methods is not always reliable. 

The main obstacle to accurate cloud motion estimation is believed to be the 
presence of mixtures of motions. One possible way to solve tliis problem is to 
use spatial frequency filtering to separate the motions of different types of 
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cloud features, since cloud size is closely related on the one hand to spatial fre- 
quency and on the othez’ hand to cloud motion. The effort to separate clouds by 
tlieir sizes tlirough frequency filtering is , however, tliwarted by tlie fact that for 
real pictures each component in the frequency domain is a complicated function 
of the features in the space domain. Cloud patterns are not perfect sinusoidal 
fimctions. More than one frequency component is always needed to represent a 
cloud element, and conversely a high frequency component may receive contri- 
butions from small cloud elements, and also from sharp corners of a large 
cloud element. To further complicate the matter, the small cloud element or 
sharp corner may not exist in the second picture with which the first picture is 
being correlated. It is not found possible to effectively separate cloud types by 
spf.tial frequency filtering. 

Another possible approach to separating cloud types for motion estimation 
purposes is to segment tlie cloud cover windows using tliresholding or pattern 
classiUcation techniques (Lo and Mohr, 1974; Lo, 1975). The brightness, the 
equivalent black body temperature in various regions of the infrared spectrum, 
and the size of clouds are all closely related to die altitude of the clouds. Since 
clouds at certain altitudes in a limited region tend to move at the same velocity, 
thresholding teclmiques could be designed to separate clouds according to their 
brightness and/or spectral equivalent black body temperatures. Picture analy- 
sis techniques wliich determine cloud size should also be investigated for the 


separation of clouds. 
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It can be concluded fi’oni tliis study that a teclinique wiiich separates clouds 
according to tlieir motions must be designed and utilized to pre-process a pic- 
ture before applying Fourier transfoi'm motion estimation teciiniques such as 
the cross -covariance and phase difference methods. The cross-covariance 
method appears to be somewhat more reliable than the phase differeiice metliod, 
and should be preferred, even though tlie latter does present the advantages of 
providing multiple estimates and relative economj'^ in computer time. 
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CAPTIONS 


Figure 1 

a. Gray levels of a pair of images simulating cloud motion; a large mass of 
relatively dim cloud undergoing a displacement (in pixels) of X = 2 and 

Y = 1, on which are superimposed smaller but brighter clouds under- 
going a displacement of X = 4, Y = 6. 

b. Power spectra of the images in a. The {0, 0) component is in the upper 
left corner, with wave number increasing downward and to the right. 

c. Cross-covariance of the images in a. Zero displacement corresponds to 
the upper left corner; X displacement increases to the right and Y dis- 
placement increases downward. 

d. Phase angles of the cross spectral density of the two images. The (0,0) 
component is in the upper left corner, with wave number increasing down- 
ward and to the right. 

e. X and Y displacement estimates derived from the cross spectral density. 

f. Scatter plot of the estimates in e. 

Figure 2 (a-f) 

Analogous to Figure 1, but for images simulating the motion of two sets of clouds 
with the same brightness but different velocities (i.e., displaced by different 
amounts) . 

Figure 3 

A pair of images selected as an example from the ATS-1 satellite, (a) The 
64 X 64 pixel portion of each image from which cloud motion is to be determined, 
(b) The surrounding area (256 by 256 pixels) in which the images in a are 
centralljf imbedded. 



GRAY LEVELS OF FIRST IMAGE GRAY LEVELS OF SECOND IMAGE 
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Figure 2b. Power spectra of the images in a. The (0,0) component is in the upper 
left corner, with wave number increasing downward and to the right. 
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Figure 2e. X and Y displacement estimates derived from the cross spectral density. 







i9 



d 



p 


,'i-- *49aT'^J- 

p 

|h 


mm 



mi 






